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00 I Abstract 

The paper deals with the nonparametric estimation problem at a given fixed 
point for an autoregressive model with unknown distributed noise. Kernel estimate 
' modifications are proposed. Asymptotic minimax and efficiency properties for pro- 



posed estimators are shown. 
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We consider the following nonparametric autoregressive model 



y k = S{x k )y k -x + £ k , l<k<n, (1.1) 



where S(-) is an unknown K — > K function, x k = k/n, yo is a constant and the noise 
random variables (6c)i<A;<n are wlTj h E£fc = and E£| = 1. 

The model (11 .ip is a generalization of autoregressive processes of the first order. In [2] 
the process f 1 1.1 1) is considered with the function S having a parametric form. Moreover, the 
paper [3] studies spectral properties of the stationary process (II. ip with the nonparametric 
function S. 
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This paper deals with a nonparametric estimation of the autoregression coefficient 
function S at a given point Zq, when the smoothness of S is known. For this problem we 
make use of the following modified kernel estimator 



1 n 

where Q(-) is a kernel function, 



n k=i 



A n = Yl QMvl-i with u k = Xk , ; 

fc=i 

d and /i are some positive parameters. 

First we assume that the unknown function S belongs to the stable local Holder class 
at the point z with a known regularity 1 < (3 < 2. This class will be defined below. We 
find an asymptotical (as n — > oo) positive lower bound for the minimax risk with the 
normalyzing coefficient 

(Pn = n w+i. (1.3) 
To obtain this convergence rate we set in (11. 2p 



i 



h — n 2 ' 3+1 and d = K n nh, (1.4) 

where k > 0, 
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lim K n = and lim — = 0. (1.5) 

n^oo n— >oo K 

n 

As to the the kernel function we assume that 

J Q(z)dz>0 and J zQ(z)dz = 0. (1.6) 

In this paper we show that the estimator (11.21) with the parameters (ll.4p - (ll.6l) is 
asymptotically minimax, i.e. we show that the asymptotical upper bound for the minimax 
risk with respect to the stable local Holder class is finite. 

At the next step we study sharp asymptotic properties for the minimax estimators 
(II. 2p . To this end similarly to [S] we introduce the weak stable local Holder class. In this 
case we find a positive constant giving the exact asymptotic lower bound for the minimax 
risk with the normalyzing coefficient (II. 3p . Moreover, we show that for the estimator (11.21) 
with the parameters (ll.4l) - (ll.5p and the indicator kernel Q = the asymptotic upper 

bound of the minimax risk coincides with this constant, i.e. in this case such estimators 
are asymptotically efficient. 

In PP, Belitser consider the above model with lipshitz condtions. The autor proposed 
a recursive estimator , and consider the estimation problem in a fixed t. By the quadratic 
risk, Belitser establish the convergence rate witout showing it's optimality. 
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Moulines and al in |9j, show that the convergence rate is optimal for the quadratic 
risk by using a recursive method for autoregressive model of order d. We note that in our 
paper we establish an optimal convergence rate but the risk considered is different from 
the one used in [9], and assymptions are weaker then those of [9]. 

The paper is organized as follows. In the next section we give the main results. In 
Section 3 we find asymptotical lowers bounds for the minimax risks. Section 4 is devoted 
to uppers bounds. Appendix contains some technical results. 



2 Main results 

Fisrt of all we assume that the noise in the model (jl.ip . i.e. the i.i.d. random variables 
(£fc)i<fc< n have a density p (with respect to the Lebesue measure) from the functional class 
V defined as 

( r+oc p+co 

V := < p > : / p(x) dx = 1 , / xp(x) dx = , 

L J —oo J — oo 

/+oo f+OO \ 

x 2 p(x) dx = 1 and / \x\ A p{x) dx < a* > (2.1) 
-oo J —oo J 

with a* > 3. Note that the (0, l)-gaussian density belongs to V . In the sequel we denote 
this density by p . 

The problem is to estimate the function S(-) at a fixed point z e]0, 1[, i.e. the value 
S(zq). For this problem we make use of the risk proposed in [5]. Namely, for any estimate 
S = S n (z ) (i.e. any mesurable with respect to the observations (yk)i<k<n function) we 
set 

TZ n (S n , S) = sup E s JS n (z ) - S(z )\ , (2.2) 

where E 5p is the expectation taken with respect to the distribution P 5p of the vector 
(yi, ...,y n ) in (jl.ip corresponding to the function S and the density p from V . 

To obtain a stable (uniformly with respect to the function S ) model (11.11) we assume 
(see [2] and [3]) that for some fixed < e < 1 the unknown function S belongs to the 
stability set 

F £ = {S 6 0^0,1] : ||5||<l-e}, (2.3) 

where H^H = sup 0<a , <1 |5"(x)|. Here C 1 [0, 1] is the Banah space of continuously differen- 
tiable [0, 1] — > R functions. 

For fixed constants K > and < a < 1 we define the corresponding stable local 
Holder class at the point Zq clS 

H w (z ,K,e) = h e T e : ||5|| < K and Q*{z ,S)<k} (2.4) 
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with (5 = 1 + a and 

x6[0,l] l x ~ z ol 

First we show that the sequence (11. 3p gives the optimal convergence rate for the functions 
S from TC^(z , K, e). We start with a lower bound. 

Theorem 2.1. For any K > and < e < 1 

iim^oo inf sup <p n K n (S n , S) > 0, (2.5) 

where the infimum is taken over all estimators. 

Now we obtain an upper bound for the kernel estimator (11.21) . 

Theorem 2.2. For any K > and < e < 1 £/ie kernel estimator (ll.2p mt/i t/ie 
parameters (II .4p - (II .6p satisfies the following inequality 

lim^oo sup Lp n TZ n (S n ,S) < oo. (2.6) 

5eW(' 3 )(2 ,E',e) 

Theorem 12.11 and Theorem 12.21 imply that the sequence (II. 3p is the optimal (minimax) 
convergence rate for any stable Holder class of regularity (3, i.e. the estimator (11.21) with 
the parameters (11.4j) - (11.6l) is minimax with respect to the functional class ( 12. 4ft . 

Now we study some efficience properties for the minimax estimators ( 11. 2ft . To this end 
similarly to [5j we make use of the family of the weak stable local Holder classes at the 
point z , i.e. for any 5 > we set 

Wg(*b,e) = {S e T £ : \\S\\ < 5- 1 and \n h (z ,S)\ < 5h?} , (2.7) 

where 

n h (z ,S) = J (S(z + uh) - S(z ))du 

and /i is given in (11.41) . 
Moreover, we set 

r(S) = l-S 2 (z ). (2.8) 

With the help of this function we describe the sharp lower bound for the minimax risks 
in this case. 

Theorem 2.3. For any 5 > and < e < 1 



hm^ inf sup r- 1/2 (S)ip n K n (S n ,S) >B\rj\ 7 (2.9) 
where rj is a gaussian random variable with the parameters (0, 1/2). 
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Theorem 2.4. The estimator (11. 2p with the parameters f ll.4p - fll.5p and Q(z) = l[-i,i] 
satisfies the following inequality 

lim^o lim^oo sup t~ 1/2 (S) ip n K n (S n , S) < E\r)\ , 

where 7] is a gaussian random variable with the parameters (0, 1/2). 

Theorems 12.31 and 12.41 imply that the estimator (11.21) . (11.411 — (11.51) with the indicator 
kernel is asymptotically efficient. 

Remark 2.5. One can show (see that for any < 5 < 1 and n > 1 

This means that the "natural" normalyzing coefficient for the functional class ( 12. 71) zs t/je 
sequence (11.31) . Theorem \2.3\ and Theorem \2.J\ extend usual the Holder approach for the 
point estimation by keeping the minimax convergence rate (II. 3p . 



3 Lower bounds 



3.1 Proof of Theorem [231 

Note that to prove (12.51) it suffices to show that 



SU P E s i(> n (S n ,S) > 

S SeHW(z ,K,s) 



(3.1) 



where 

ip n (S n ,S) = Lp n \S n (z ) - S(z )\ . 

We make use of the similar method proposed by Ibragimov and Hasminskii to obtain 
a lower bound for the density estimation problem in [5|. First we chose the correspond- 
ing parametric family in JvP> (z , K , e) . Let V be a two times continuously differentiable 
function such that j\ V(z)dz > and V{z) = for any \z\ > I. We set 



u_ l x - z Q 



<p n V h 

where ip n and h are defined in (11. 3ft and ( 11. 4ft . 

It is easy to see that for any z — h<x<z + h 



(3-2) 



< 



i"i 

| It | 



r 



v" 



x — z 

h 

x - z 



h 



- V(0) 

< \u\V" \x — Zq\ 
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where V" = max| x |<i |V"(^)|. Therefore, for all < u < u* = K/V" we obtain that 



SUp : < A . 

zo— h<x<zo+h \ x z 0\ 



Moreover, by the definition ( 13. 2p for all x > £ + /i 

S„0) = S u (z + /i) = and S u (x) = S u (z - h) = 
for all x < z — /i respectively. Therefore, the last inequality implies that 

sup n*(z Q ,S u ) < K, 

\u\<u* 

where the function fl*(z , S) is defined in ( 12. 41) . 

This means that there exists n Ke > such that S u E H^\z , K,e) for all \u\ < u* 
and n > n K £ . Therefore, for all n > n Ke and for any estimator S n we estimate with below 
the supremum in (13. ip as 

SU P E S , Po i>n{Sn,S)> SUp E s 1p n (S n ,S u ) 

S&iW{z ,K,z) \u\<u* 

rb 



>Y b J_ b E s u ^n(S n ,S u )du (3.3) 



for any < b < u* . 

Notice that for any S the measure Ps, Po is equivalent to the measure Po Po ; where 
Po,p ls the distribution of the vector (yi, . . . , y n ) in (jl.ip corresponding to the function 
S = and the gaussian (0,1) noise density p , i.e. the random variables (yx,...,y n ) are 
i.i.d. A/"(0, 1) with respect to the measure Po, Po - in the sequal we denote Po, Po by P. It is 
easy to see that in this case the Radom-Nikodym derivative can be written as 

HP 2 

with 

^n = —Y] V 2 (uk)t,l-! and r] n = V V(u k ) £ fc _i & . 

Pit! SnPn^ 

Through the large numbers law we obtain 

k* i 

P - lim e n = lim \ V V2 ^l-i = [ V 2 (u)du = a 2 , 

ra— >oo n-*oo nil z — ' / , 

where 

k* = [nz — nh] + 1 and k* = [nz + nh] . (3-4) 
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Here [a] is the integer part of a. 

Moreover, by the central limit theorem for martingales (see [I] and [6j), it is easy to 
see that under the measure P 

r] n =>- (0, 1) as n — > oo . 

Therefore we represent the Radon-Nykodim density in the following asymptotic form 



where 



Pn (u) = e^Vn-^+rn 



P — lim r n = . 

n— »oo 



This means that in this case the Radon-Nikodym density (p n (u)) n>1 satisfies the L.A.N, 
property and we can make use the method from theorem 12.1 of [8] to obtain the folowing 
inequality 

lim^^inf ^ J E SujPQ ip n (S n ,S u )du > I(b,a), (3.5) 



where 



max(l,b-y/b) a 
I(b,cr) = — : -—= I e a du 



and < b < u* . Therefore, inequalities (13.31) and ( 13. 5ft imply ( 13.1ft . Hence Theorem 12. 1L 



□ 



3.2 Proof of Theorem Q 

First, similarly to the proof of Theorem 12.11 we choose the corresponding parametric 
functional family S uv {-) in the form (13.21) with the function V = V u defined as 

K(x) = v- 1 j°° Q u (u)g 0^-) du , 

where Q u {u) = l{\ u \<i-2u} + 21{i-2u<\u\<i~u} with < v < 1/4 and g is some even nonneg- 
ative infinitely differentiable function such that g(z) = for \z\ > 1 and J_ x g(z) dz = 1. 
One can show (see [5]) that for any b > 0, < 5 < 1 and < v < 1/4 there exists 
n * = n *{b, 5,v) > such that for all \u\ < b and n > 

Therefore, in this case for any n > 

<p n sup r^ 2 (S)TZ n (S nl S)> sup r- l / 2 (S)E SjP J n (S n ,S) 

> T *( n ' 6 )^ J E S u ^n(S n ,S UjU )du. 
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where 

r>,6)= inf r-^iS^). 

\u\<b 

The definitions (I2.8P and (13.21) imply that for any b > 

lim sup \t(S uu ) — 1| = 0. 

™->°°|m|<6 

Therefore, by the same way as in the proof of Theorem 12.11 we obtain that for any b > 
and < v < 1/4 

!im„^oo ir if sup r" 1/2 (5) if n TZ n (S n , S) > I(b, a u ) , (3.6) 

where the function I(b, a v ) is defined in ( 13.51) with o 2 v = f* V^(u) du. It is easy to check 
that a 2 u — > 2 as z/ — > 0. Limiting 6 — > oo and — > in (13.61) yield the inequality (12.91) . 
Hence Theorem 12.31 
□ 



4 Upper bounds 

4.1 Proof of Theorem 12^1 

First of all we set 



~ A — 1 

K = -j and A n = -~- 1 (X >K } . (4.1) 



Now from (ll.2p we represent the estimate error as 

1 - 1 



S n (z ) - S(z ) = -S(zo) l {An<Kj + — A n C n + —A n B n , (4.2) 

n n Yn Yn 



with 



j- I2k=iQ( u k)yk-itk , D Efc=i <5K) - s ( z o))y k ~i 

Cn = and B n = . 

Note that, the first term in the right hand of (14.21) is studied in Lemma [A. 31 To estimate 
the second term we make use of Lemma IA.2I which implies directly 

lim^oo sup sup E 5)P C n < oo 

SeH^(z ,K,e) pev 



and, therefore, by (1A.8|) we obtain 



lim^oo sup sup E 5jp \A n \ \( n \ < oo . 
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Let us estimate now the last term in the right hand of (14. 2 j) . To this end we need to show 
that 

lim^oo sup sup E StP B 2 n < oo . (4.3) 

SeHW(z ,K,e) pev 

Indeed, putting r k = S(xk) — S(z ) — S(zo)(xk — zq) by the Taylor Formula we represent 
B n as 

B n = —S(z )B n + — B n , 

where B n = J2k=i Q( u k)u k yl_ 1 and B n = J2k=i Q( u k) r k yl-i- We remind that by the 
condition (I1.6P J 1 uQ(u)du = 0. Therefore through Lemma [A .21 we obtain 

h 2 

lim — sup sup E 5 B 2 = . 

n ^°° SeHW(z ,K,B) p&V n 

Moreover, for any function 5* G T-C^(z , K, e) and for k^ < k < k* (k^ and k* are given in 

(E3D) 



r k 



S(u) — S(z ) ) du 



< K\x k -z f < Khl 3 = Kifi^ 1 



i.e. B n < (f n A n . Therefore, by Lemma lA. 2 1 

lim^oo sup sup E Sp B 2 n < oo . 

SeHW(z ,K,e) p&V Vn 

This implies (14.31) . Hence Theorem 12 .2[ □ 
4.2 Proof of Theorem 12^1 

Similarly to Lemma A. 2 from [5] by making use of Lemma [A. II and Lemma [A. 21 we can 

show that i 

t(S) 

-±-± ( n =^ jV(o, 1) as n -> oo 

uniformly in S G T e and p E V. Therefore, by Lemma IA.2I we obtain that uniformly in 
S G T e and p G V 

T- l ' 2 {S)A n C n AT (0,1/2) as n - oo . 

Moreover, by applying the Burkholder inequality and Lemma IA.2I to the martinagale ( n 
we deduce that 

lim^oo sup sup E S p < oo . 

SeHW(z ,K,e) pev 
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Therefore, inequality (I A. 81) implies that the sequence (A n C n )n>i i s uniformly integrable. 
This means that 



lim sup sup 

SeHW(z ,K,e) p£V 



n— >oo 



T- 1/ \s)E s \A n a-m 



0. 



where rj is a gaussian random variable with the parameters (0, 1/2). Now to finish this 
proof we have to show that 

lim lim^oo sup sup E 5p 5^ = 0. (4.4) 

MIW) pep 

Indeed, by setting f s (u) = S(z + hu) — S(z ) we rewrite B n as 

k* 

where n 2 

QnU,S) = ^ ^s) J j^du 

and £l h (z , S) is defined in (12.71) . The definition (12.81) implies that for any S E T £ 

£ 2 <t(S)<1. (4.6) 
From here by the definition (12. 7p we obtain that 

\ B n\ < t Pn\e n {fs, S )\ + -t- 

Moreover, for any S G Ug^(z ,e) the function f s satisfies the following inequality 

ll/sll + ll/sll<^- 

We note also that (f n h 2 — > as n — > oo. Therefore, by making use of Lemma [A.2I with 
R = h/5 we obtain (T4T4D . Hence Theorem [23 □ 

5 Appendix 

In this section we study distribution properties of the stationary process (11.11) . 

Lemma A.l. For any < e < 1 the random variables ( 11 .lft satisfy the following moment 
inequality 

m* = sup sup sup sup E s < oo . (A.l) 

n>i o<fc<n ser P eP 
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Proof. One can deduce from (11 .ip with S G T £ that for all 1 < k < n 

Vt < ^(1 - e) k \y \ + E (! - £ ) k ~ J felj < % 4 + 8 (E ( X " ^ fel 
Moreover, by the Holder inequality with g = 4/3 and p = 4 

J'=l 

Therefore, for any p E V 



E s , p y 4 k <8\y \ 4 + -a*. 

Hence Lemma [A. 11 □ 

Now for any K > and < e < 1 we set 

Sjf^^er, : \\S\\<K}. (A.2) 

Lemma A.2. Let the function f is two times continuously differentiable in [—1, 1], such 
that f(u) = for \u\ > 1. Then 

1 

R>0 \ Kh ) \\f\W<R S&e K ,s P&V 



lim sup frtl , N9 sup sup snpE Sp g n (f,S) < oo , (A. 3) 



where Wf^ = \\f\\ + ||/|| and g n (f,S) is defined in (J45]). 
Proof. First of all, note that 



k=l 

where 

k* k* 



T n = E -/"M^ and a « = E ~~ /K-i)) vl-i - /K*) V, 

k=k* k=k t 

with fc* and k^ defined in ( 13. 4p . Moreover, from the model (II. ip we find 

k* 

T n = W) + E /W^t-l + M n , 



k=k» 

where 



Lif) = E ^^ fc ) and Mn = E ^( Mfc ) (2 5'(a; fc )y fe _i^ + r? fe ) 



k=k, k=k* 
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with T]k = ^1 — 1. By setting 

k* 



k=K 



k=k* 



we get 



C n =Yl ( s2 ( x k) - S 2 (z )) f(uk) yl-i and D n = ^ /(«fe)(l/*_i - 3/fc) 

1 - 1 /„(/) , 1 A n 



2 «■ 



is) & 



(A.5) 



with A n = M n + C n + S 2 (z ) D n . Moreover, taking into account that (p 2 = nh we obtain 

k* 



' r. 



f(t)dt 



T ( f) f 1 p u k r 1 

- / f(t)dt+Y, / fMdt- / 

1 k=k* ^ u k-i •'-I 

=e r (f(u k )-f(t))dt+ [ k * f(t)dt~ i /(/)(//-. 



We remind that ||/|| + ||/|| < R. Therefore 

k* 



u fc*-i 



1 Z" 1 



< 



i2 



Taking this into account in (IA.5I) and the lower bound for t(S) given in (14. 6p we find that 



f(t)dt 



< 



e 2 \ nh nh nh nh 



(A.6) 



Note that the sequence (M n ) n>1 is a square integrable martingale. Therefore, 

2 -, k* 



E 



S.p 



1 

n/i 



~f^2 E s, P ( 2 S(x k )y k -i^k + VkY 



< 



k=k t 

4R 2 (4 v / m* + a*) 
nh 



where m* is given in ( 1A.1I) . Moreover, taking into account that \S(xk) — S(zq)\ < L\xu — z \ 
for any S G 6 L s and that k* — k^ < 2nh we obtain that 

fc* 

7^ E ^ - ^ E l( ^ (Xfc) " 52 (^))i 2 / 2 (^) E ^^ 4 -i 
^ ' k=k t 

< 16 R 2 L 2 m* h 2 . 



Let us consider now the last term in the right hand of the inequality (IA.6I) . To this end 
we make use of the integration by parts formula, i.e. we represent D n as 

k* 

D n = ~~ f( u k-i)) Vk-i + f( u k*-i) vt-i - f( u k*) vl* ■ 
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Therefore, taking into account that ||/|| + ||/|| < R we obtain that 



E s , p D 2 n < 3R 2 E s>p ( A £ yU + yl +yi_ 1 ) < 18 R 

k=k* 



2 * 

m 



By the same way we estimate the second term in the right hand of (IA.4I) . Hence Lemma fA.2[ 
□ 

Lemma A. 3. The sequences (^4 n ) n>1 and (^4 n ) n>1 defined in (14.11) satisfy the following 
properties 



lim — sup sup P 5 (A n <K n )<oo (A.7) 



and 



lim sup sup E s A < oo . (A. 8) 



Proof. It is easy to see that the inequality (1A.7|) follows directly from Lemma IA.2I We 
check now the inequality flA.8j) . By setting 7^ = e~ 2 J Q{u)du we get 

Es ' p ^ = 4 / t3 Ps ' P ^ n ~ ' ln > " n ) ^ 
<4 f" t 3 P SiP (g n (Q,S)+ % <r 1 ) dt 
2\ 4 1 



^ Ur +74 p ^(I^(Q^)I>t*/2). 

\7*/ K n 

By making use of Lemma IA.2I with the condition (11.51) we obtain the inequality (1A.8I) . 
□ 
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